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ABSTRACT 


The  Johns  Hopkins  University  Applied  Physics  Laboratory  is  developing  a 
technique  that  uses  aggregate  system  performance  data  to  explore  multi-dimensional 
trade  spaces  among  military  capabilities.  The  process  is  used  to  examine  the  operational 
value  of  trading  systems  within  capabilities  and  capabilities  within  warfare  areas. 

Our  methodology  models  joint  military  performance  and  examines  capabilities 
associated  with  combining  systems.  Evaluation  of  tactics,  techniques  and  procedures, 
measures  of  performance,  and  measures  of  outcome  is  common  within  the  Department  of 
Defense;  developing  an  over-arching  predictive  tool  is  desired.  Trade  space  analysis, 
response  surface  methodology,  and  optimization  are  not  new  approaches;  the 
combination  of  these  into  an  orchestrated  process  is. 

A  variety  of  capabilities  each  with  a  range  of  performance  values  combine  to 
produce  results  in  an  area.  A  specific  capability  determines  the  end  state  resulting  of 
combination;  the  desired  end  states  may  differ  from  campaign  to  campaign.  All  possible 
combinations  of  contributing  systems  describe  the  capability. 

An  optimization  process  explores  the  performance  parameters  for  all  systems 
simultaneously,  aggregates  system  values  to  define  capabilities,  and  searches  for  the 
optimal  combination  of  capabilities.  The  optimization  process  is  provided  response 
surface  approximations  over  which  to  search  for  optimal  regions. 
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1  PROJECT  OVERVIEW 


1.1  STUDY  OBJECTIVE 

Developing  an  optimization  technique  that  uses  system  performance  data  to 
explore  multi-dimensional  trade  spaces  among  military  capabilities  is  beneficial  to 
military  sponsors.  This  study  develops  an  understanding  of  trade  space  among  military 
platforms  and  their  combined  contribution  to  warfare  capability.  This  understanding  is  to 
be  exploited  by  the  use  of  response  surfaces  that  map  configured  output  from  military 
models  (simulation  based  or  not)  into  multidimensional  space.  Once  mapped,  the 
response  surface  yields  useful  comparisons  of  platform  contributions  toward  capabilities 
supporting  military  campaigns.  Ultimately,  the  goal  is  to  integrate  multi-dimensional 
trade  spaces  into  a  common  analytical  picture  representing  a  warfare  area.  Overall,  this 
work  is  a  proof  of  concept  the  goal  of  which  is  to  develop  a  tool  with  analytic  rigor  to 
integrate  military  capabilities  defining  warfare  areas. 

1.1.1  Integration 

Military  analysis  focuses  on  warfare  from  various  perspectives  and  at  different 
echelons.  It  is  critical  to  take  an  integrated  perspective  of  joint  assets  as  warfare  areas 
generally  separated  by  air-land-sea  become  more  closely  associated  and  merge  for 
combined  effect  on  the  battlefield.  The  joint  commander  must  make  “all  arms”  decisions 
that  optimize  the  use  of  joint,  even  international,  assets.  Military  sponsors  are  interested 
in  analysis  that  directly  addresses  interactions  at  the  capability  level.  These  are  the 
interactions  of  platforms  but  with  a  decided  focus  on  the  overall  effect  or  synergy 
associated  with  a  particular  combat  mix.  The  study  plan  depicted  in  Figure  1.1  was  used 
to  develop  a  methodology  by  gathering  data  representative  of  military  conflict  on  a  large 
scale,  developing  mathematical  approximations  of  the  response  variables,  and  optimizing 
across  these  approximations  to  derive  the  most  beneficial  combination  of  assets  to 
achieve  the  desired  outcome. 
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Transformation  Function  Approximations 
Feasible  Solution  Space 


Figure  1.1.  Structure  of  the  research  plan. 


1.1.2  Analysis 

Response  surface  methodology  (RSM)  is  a  predictive  model  that  forecasts 
performance.  We  approximated  this  multi-dimensional  surface  by  evaluating  tactics, 
techniques,  and  procedures  while  observing  their  effect  via  measures  of  effectiveness 
(MOE).  This  surface  approximates  the  entire  feasible  space.  The  military  commander  can 
further  constrain  a  scenario/campaign  or  region  of  interest  to  a  specific  portion  of  the 
surface.  Within  this  region,  the  commander  will  understand  the  marginal  value  of 
platforms,  tactics,  and  objectives.  These  may  be  weighed  against  each  other  or  traded  to 
produce  a  range  of  desirable  outcomes. 

Step  1  (Model  Selection  and  Data  Configuration) 

.  The  Joint  Integrated  Contingency  Model  (JICM)  was  used  to  develop  data  for 
analysis.  Input  parameters  were  selected  and  configured  on  the  basis  of  their  significance 
of  their  effect  and  commonality  across  platforms. 

Step  2  (RSM  Configuration/Trade  Space  Development) 

Experimental  design  is  used  to  develop  a  complete  picture  of  the  simulation 
output.  Data  analysis  produces  polynomial  approximations  of  these  responses  creating 
response  surfaces.  We  used  second-order  approximations  although  higher-order 
predictors  could  be  produced. 

Step  3  (Trade  Space  Integration) 

The  surface  approximations  are  combined  and  analyzed,  producing  comparisons 
between  effects  associated  with  all  joint  assets  contributing  to  the  capability.  An 


1-2 


optimization  tool  was  developed  that  considers  the  response  surfaces  associated  with 
every  measure  of  outcome. 

Step  4  (Capability  Assessment) 

The  end  result  is  rapid  assessment  across  scenarios  including  analytic  excursions 
exploring  various  campaign  options,  and  these  excursion  results  can  be  produced  in 
minutes. 

1.1.3  Joint  Sponsors 

This  type  of  analysis  is  being  conducted  throughout  the  Department  of  Defense. 
The  U.S.  Army  is  performing  “Value  Added  Analysis”  to  evaluate  their  modernization 
plan  at  the  Center  for  Army  Analysis,  Ft.  Belvoir,  VA.1  The  US  Air  Force’s  Air  Combat 
Command  is  analyzing  effects-based  modernization  at  Langley  Air  Force  Base,  VA.2  The 
U.S.  Navy  Pacific  Command  and  Air  Force  Center  for  Studies  and  Analysis  has  voiced 
interest  in  this  type  of  process. 

1.2  CONCEPT  OF  WARFARE  INTEGRATION 
1.2.1  Background 

Extensive  applications  of  RSM  are  seen  in  the  industrial  world,  particularly  in 
situations  where  several  input  variables  potentially  influence  some  performance  measure 
or  quality  characteristic  of  a  process.  This  performance  measure  or  quality  characteristic 
is  called  the  response.  It  is  typically  measured  on  a  continuous  scale,  although  attribute 
responses,  ranks,  and  sensory  responses  are  not  unusual.3  RSM  is  a  collection  of  tools  for 
understanding  the  nature  of  a  relationship  between  independent  variables  and  some 
measure  of  performance.  The  three  key  elements  of  RSM  are  design  of  experiments, 
empirical  modeling,  and  optimization.  The  input  variables  are  considered  independent 
and  are  controlled  by  the  investigator.  Statistical  and  mathematical  modeling  is  used  to 
develop  an  approximate  relationship  between  a  response  variable  (Z)  and  input  variables 
(Si,  S2,  ••••  Sic)-  Optimization  methods  are  used  for  finding  levels  of  the  input  variables 
that  produce  the  desired  value  of  the  response.  Modeling  curvature  is  very  important 
when  the  objective  is  to  find  an  optimum  response.3  The  functional  form  is 

z=f&,S2,  ....|k)  +  s, 

whore  8  represents  sources  of  variability  not  accounted  for  in  the  function;  this  can  be 
measurement  error  and  background  noise.  The  investigator  assumes  that  there  is  a 
mathematical  relationship  between  the  variables,  but  the  specific  relationship  is  unknown. 
Usually  a  lower-order  polynomial  in  a  region  of  the  independent  variable  space 
represents  the  response  as  a  surface  in  space. 

If  the  fitted  surface  is  an  adequate  approximation  of  the  true  response,  then 
analysis  of  the  fitted  surface  is  equivalent  to  analysis  of  the  actual  system.  This  allows  the 
investigator  to  estimate  the  response  using  the  surface  rather  than  conducting  additional, 
costly,  time-consuming  experiments.  The  investigator  simply  reads  the  response  from  the 
graph. 
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Figure  1.3.  Example  response  surface.  The  vertical  axis  is  a  predicted  result  of 
combining  the  two  horizontal  variables. 


1-2.2  Experimental  Design  Applied  to  Warfare  Integration 

Joint  warfare  involves  many  combat  platforms  with  associated  command  and 
control  equipment.  The  argument  between  aggregate  and  disaggregate  representation  has 
significant  impact  on  the  use  of  this  research  and  this  methodology.  The  approach  here  is 
a  fairly  standard  representation  of  combat  platforms  as  systems-of-systems  where  each 
major  system  is  composed  of  a  list  of  minor  systems  contributing  to  its  performance.  A 
computer  model  generates  simulated  combat  results  representing  the  major  systems 
performance  recorded  as  measures  of  outcome  (MOO).  Figure  1.4  represents  the 
conceptual  data  development.  A  matrix  of  major  systems  [5]  is  established,  and  each 
major  system  is  defined  by  the  value  of  its  sub-systems  [Val].  This  S  matrix  is  provided 
to  a  computer  model  that  transforms  the  input  matrix  into  simulated  combat  results  and 
records  specified  measures  of  outcome  into  a  matrix  [M\.  Ultimately,  the  starting 
conditions  for  the  systems  can  be  compared  graphically  to  the  MOO  using  a  response 
surface.  The  major  systems  are  the  factors  for  the  experiment  because  they  are 
established  by  the  investigator. 
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Figure  1.4. 


The  transformation  of  matrix  input  into  a  response  matrix.4 


Discussed  here  briefly  are  the  mathematics  behind  this  process,  where  each  major 

system  <Sk  is  represented  by  §k.  The  factors  £i,  \2 . are  called  natural  variables.  They 

represent  the  natural  units  of  measure  (e.g.,  when  measuring  velocity,  the  value  of  £1  may 
be  60  miles/hour  [mph]).  It  is  better  to  use  coded  variables X\,X2,  ...Xkofthe  form 


max(^)  +  mm(^) 


max(^)-min(^) 

2 


The  distribution  of  values  for  each  variable  is  “normalized”  to  fall  between  -1 
and  1  with  mean  zero  and  have  the  same  standard  deviation  as  Fundamental  to 
choosing  a  design  that  minimizes  estimation  variance  in  the  response  is  that  values  are 
also  orthogonal.  Varying  only  one  factor  at  a  time  can  establish  knowledge  of  the  effect 
of  one  factor  when  the  others  are  held  constant. 

It  is  desirable  to  measure  interactions  between  variables.  Interaction  implies  that 
the  effect  of  one  factor  on  the  response  does  not  remain  the  same  for  different  levels  of  a 
second  factor.  To  obtain  information  on  interactions,  the  levels  of  each  factor  are  varied 
and  all  possible  combinations  are  considered;  this  is  called  a  factorial  experiment.  The 
value  of  a  factorial  experiment  is  that  it  looks  at  several  factors  simultaneously  and 
allows  various  effects  to  be  estimated,  thus  allowing  the  investigator  to  draw  conclusions 
over  a  wider  range  of  conditions. 
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One  disadvantage  of  factorial  experiments  is  that  the  number  of  treatment 
combinations  increases  rapidly  as  the  number  of  factors  and/or  levels  increases.  Three 
variables  examined  at  two  levels  may  be  easily  considered  at  every  possible  combination 
(23  =  8)  while  ten  variables  at  two  levels  exponentially  increases  the  requirement  (210  = 
1024).  One  method  is  to  consider  only  a  subset  of  all  possible  treatment  combinations,  a 
fractional  factorial  Factorial  designs  collect  data  at  the  vertices  of  a  polyhedron  in  p- 
dimensions  (p  is  the  number  of  factors  being  studied).  Fractional  factorial  designs  collect 
data  from  a  specific  subset  of  all  possible  vertices.  It  is  reasonable  to  assume  that  most 
systems  are  largely  affected  by  main  effects  and  low-order  interactions.  This  “sparsity  of 
effects  principle”  is  the  underlying  assumption  that  higher-order  interactions  are  usually 
unimportant  when  the  experiment  is  considered  as  a  whole.  Thus,  by  selecting  the  proper 
combinations,  the  investigator  can  determine  how  two  or  three  factors  interact  but 
sacrifices  the  ability  to  determine  higher-order  interactions.5 


2  SIMULATING  JOINT  WARFARE  CAPABILITY 


2.1  ESTABLISHING  THE  ENVIRONMENT 

2.1.1  Overall  Campaign 

A  campaign  scenario  was  established  in  order  to  exercise  a  wide  variety  of 
military  platforms  in  diverse  environments  and  situations.  Two  theaters  were  selected: 
Northeast  Asia  (NEA)  and  Southwest  Asia  (SWA).  Existing  NEA  and  SWA  scenarios 
were  modified  to  minimize  development  time.  The  selection  of  these  two  theaters 
provided  several  experimental  and  analytical  advantages:  (1)  The  diversity  of  terrain  and 
weather  required  the  military  equipment  to  operate  across  a  wider  range  of  capabilities; 
(2)  the  opposition  forces  are  very  different  in  their  composition,  tactics,  and  equipment; 
and  (3)  databases  containing  these-theaters  exist  and  have  been  validated. 


Joint  Forces 

The  basic  structure  of  U.S.  and  Allied  forces  in  each  of  these  theaters  was  kept  the 
same.  The  “real-world”  strength  of  forces  was  modified  in  an  effort  to  produce  larger 
variance  in  their  interactions  and  gauge  overwhelming,  as  well  as  unsatisfactory,  power 
projection.  The  combat  plans  focused  on  maintaining  consistency  of  the  experimental 
design  in  order  to  minimize  spurious  variation  in  outcomes.  Thus,  even  when  additional 
air  or  combat  power  was  available  in  the  theater,  the  concept  of  operation  and  scheme  of 
maneuver  remained  the  same. 

Threat  Force  Representation 

In  each  case,  the  opposition  forces  were  to  invade  the  neighboring  country.  The 
United  States  and  Allies  were  to  conduct  a  “Halt  Operation”  to  deny  the  objective  and 
stop  the  enemy  advance. 

2.1.2  Analysis  Focused  on  Two  Theaters 
SWA  Theater 

The  SWA  theater  concept  is  shown  in  Figure  2.1. 
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Figure  2.1.  The  SWA  theater  concept. 

The  key  to  Iraqi  success  in  this  scenario  is  a  rapid  ground  advance  that  would 
allow  forces  to  threaten  the  port  at  AdDammam  and  force  the  U.S.  forces  to  adopt  a 
different  operational  plan  rather  than  reinforce  through  that  port.  For  simplicity  we  did 
not  simulate  the  deployment  of  U.S.  forces  to  the  theater  but  assumed  that  they  were  in 
place  on  D-Day.  Since  only  the  halt  phase  was  simulated,  this  assumption  is  not 
unreasonable. 

SWA  Ground  Force  Structure 

In  order  to  minimize  the  warning  to  Blue,  Iraq  used  only  nine  divisions  out  of  the 
entire  ground  force  for  the  attacks  in  this  scenario.  The  Blue  defense  consists  of  eight 
heavy  brigades  of  the  Gulf  Cooperation  Council  (GCC)  and  the  U.S.  brigade  of  varying 
capability  depending  on  the  case  (Table  2.1).  In  the  medium  case,  the  U.S.  brigade 
consisted  of  200  tanks,  200  Infantry  Fighting  Vehicles  (1FV),  200  armored  personnel 
earners  (APC),  and,  200  heavy  armored  vehicles  in  the  armor  category.  The  infantry 
consisted  of  140  short-range  anti-armor  missiles  and  1666  small  arms.  The  artillery 
category  was  represented  by  100  self-propelled  artillery  pieces.  Changing  the  armor, 
infantry,  or  artillery  variable  to  the  low  setting  cut  these  numbers  in  half  while  the  high 
setting  increased  them  by  50%. 
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Table  2.1.  SWA  ground  summary.6 


•  Iraq 

-  9  heavy  divisions  (6  Armor,  3  Mech) 

.  GCC 

-8  heavy  brigades  (4  Armor,  4  Mech) 

•US 

- 1  Brigade  varying  in  Armor,  Inf,  and  Arty 
- 1, 2,  or  3  Patriot  Bn  (3  Patriot  II,  1  Patriot  III  each) 
- 1 , 2,  or  3  Aviation  Bn  (20  Apache-D  each) 


In  addition,  the  number  of  U.S.  Patriot  and  aviation  units  (attack  helicopters) 
varies  with  case.  The  Patriot  battalions  have  six  Patriot  II  launchers  and  two  Patriot  m 
launchers.  There  are  20  Apache  helicopters  in  each  battalion,  and  the  low,  medium,  and 
high  levels  of  this  variable  meant  that  either  one,  two,  or  three  battalions  were  sent. 

SWA  Air  Force  Structure 

The  Iraqi  air  force  is  about  a  50/50  mix  of  modem  and  older-generation  fighter 
aircraft  (Table  2.2).  Pilot  proficiency  is  relatively  low,  however,  so  overall  performance 
is  modest,  particularly  in  the  air-to-ground  role. 

The  GCC  air  force  consists  of  relatively  modem  aircraft.  Pilot  proficiency, 
especially  in  the  air-to-ground  role,  is  well  below  U.S.  standards. 

Table  2.2.  SWA  air  summary.6 

•  Iraq 

-  325  fighters 

.  GCC 

-484  fighters 

•  US 

-  63, 126,  or  189  fighters  (USAF) 

-40  bombers 

-  54, 108,  or  162  fighters  (USN) 
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The  U.S.  aircraft  deployment  consists  of  40  heavy  and  medium  bombers  and  a 
variable  number  of  U.S.  Air  Force  (USAF)  and  U.S.  Navy  (USN)  fighter  aircraft.  These 
deployments  mirror  the  50%  of  the  medium  value  for  the  low  case  and  150%  of  the 
medium  value  for  the  high  case  as  was  seen  for  the  U.S.  ground  deployments.  Note  that 
some  U.S.  Navy  aircraft,  even  in  this  relatively  low  threat  environment,  must  be 
employed  for  fleet  air  defense  rather  than  offensive  missions  in  the  theater. 

NEA  Theater 

The  success  of  a  North  Korean  attack  depends  on  forcing  a  political  settlement 
before  the  United  States  can  bring  reinforcements  to  bear  to  reverse  any  Red  gains.  Thus, 
as  in  the  NEA  scenario,  there  is  a  premium  on  rapid  early  success.  The  SEA  theater 
concept  is  shown  in  Figure  2.2. 


Figure  2.2.  The  NEA  theater  concept. 


NEA  Ground  Force  Structure 

Unlike  the  SWA  scenario,  the  deployed  U.S.  brigade  has  only  a  supporting  role  in 
the  halt.  The  U.S.  aviation,  both  fixed  and  rotary  wing,  and  to  a  lesser  extent  the  air 
defense  will  affect  the  halt  conditions  in  the  simulation. 

While  attempting  to  achieve  some  degree  of  surprise  and  hence  rapid  success, 
North  Korea  does  not  mobilize  its  entire  army  but  attacks  with  a  subset  of  ground  forces. 
For  the  halt  phase  of  this  rapid  response  scenario.  South  Korea  defends  with  available 
active  duty  forces.  Reserve  forces  might  play  a  role  in  later  phases  of  the  combat  but  only 
the  halt  phase  was  simulated  so  those  forces  had  no  role  in  this  simulation.  The  U.S. 
forces  mirror  those  used  in  the  SWA  case  (Table  2.3). 


Table  2.3.  NEA  ground  summary.6 
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•  North  Korea  for  each  of  3  axes 

-  1st  echelon,  ID,  IB,  ArtyB,  and  2  MRLB 
-2nd  echelon,  ArmB,  MXB,  MRLB,  ArtB 
-3rd  echelon,  MXB,  ArtB 

•  South  Korea  for  each  of  3  axes 

-  2  ID,  MRD,  MXD,  2  ArtB,  FAG 
.  US 

- 1  Brigade  varying  in  Armor,  Inf,  and  Arty 

- 1 , 2,  or  3  Patriot  Bn  (3  Patriot  II,  1  Patriot  III 
each) 

- 1, 2,  or  3  Aviation  Bn  (20  Apache-D  each) 


NEA  Air  Force  Structure 

The  North  Korean  air  force,  while  numerically  significant,  consists  largely  of 
older-generation  aircraft.  North  Korean  pilots  get  relatively  few  flying  hours  for  training, 
so  proficiency  is  marginal. 

The  South  Korean  air  force  represents  a  mix  of  older-generation  and  more 
modem  aircraft.  Pilot  proficiency  is  much  better  than  that  of  their  North  Korean 
counterparts. 

As  in  SWA,  the  U.S.  aircraft  deployments  consist  of  40  heavy  and  medium 
bombers  and  a  variable  number  of  fighter  aircraft  (Table  2.4). 

Table  2.4.  NEA  air  summary.6 

•  North  Korea 

-  780  fighters 

•  South  Korea 

-  353  fighters 

.  US 

-40  bombers 

-63, 126,  or  189  fighters  (USAF) 

-  54, 108,  or  162  fighters  (USN) 


2.2  SIMULATION  SELECTION 
2.2.1  Criteria 

The  criteria  for  simulation  selection  were  straightforward:  (1)  The  simulation 
must  represent  joint  U.S.  military  operations;  (2)  it  must  be  affordable  to  acquire  and  use; 
and  (3)  it  must  run  quickly  in  order  to  simulate  multiple  iterations  of  the  campaign.  In 
addition,  it  was  desirable  for  the  simulation  to  be  available  to  JHU/APL  for  future  use,  be 
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validated/accredited  by  the  Department  of  Defense,  and  have  some  pedigree  of  use 
among  military  analysts. 

2.2.2  The  Joint  Integrated  Contingency  Model 

The  JICM  is  a  global  analysis  and  war-gaming  system  developed  at  RAND  under 
the  sponsorship  of  the  Director  of  Net  Assessment  in  the  Office  of  the  Secretary  of 
Defense.  It  encompasses  the  strategic  and  operational  levels  of  land,  air,  and  sea  warfare 
with  a  global  set  of  models  and  databases.  JICM  is  a  deterministic  model  with  a 
minimum  4-hour  time  step.  Brigade-  and  division-sized  ground  units  are  organized  into 
corps-level  commands  that  maneuver  on  a  predefined  network  with  variable  terrain  type 
and  width.  The  NEA  movement  network  is  shown  in  Figure  2.3.  Air  combat  is  organized 
around  an  air  tasking  order  that  packages  sorties  at  the  beginning  of  the  day  to  execute 
across  the  designated  time  periods.  Maneuver  combat  is  adjudicated  by  totaling  weapon 
scores  and  calculating  loss  rates  and  movement  from  force  ratios.9 


Figure  2.3.  JICM  ground  model  movement  network.9 


2.3  EXPERIMENTAL  DESIGN 


2.3.1  Establishing  the  Input  Parameters 

In  this  research  we  want  to  approximate  a  quadratic  model  for  13  coded  input 
variables  and  28  response  variables.  Table  2.5  lists  the  13  variables  with  their  associated 
natural  and  coded  levels. 

Table  2.5.  Values  for  varying  parameters. 


2-6 


input 

Variables 

Description 

Natural  (g). 

Coded  Levels 

MR 

Red  Advance  Rate 
multiplier 

5. 

HBH 

-1,0,1 

iiriiiiiii 

mm 

^2 

-1,0,1 

PAT 

U,S.  air/missile  defense 
battalions  Patriot  I  and 
PatriotlH 

& 

3,6,9 

1,2,3 

Xi 

-1,0, 1 

ARM 

U.S.  IFV,  APC,  and  armor 
per  brigade 

HH 

X, 

-1,0,1 

INF 

5s 

mm&m 

mtmmimwm 

X5 

-1,0,1 

ARTY 

it 

50, 100, 150 

Xt 

-1,0,1 

HELD 

57 

20,40,60 

Xj 

-1,0,1 

TLAM 

U.S.  TEAM  used  to  attack 
airbases  . 

58 

200,400, 600 

X, 

-1,0,1 

JSOW 

59 

** 

X9 

-1,0, 1 

ATOM 

U.S.  ATAGM  quantity 

§10 

30,60,90 

Xw 

mm 

.  USAF 

USAF  tighter/bomber 
aircraft  deployed 

In 

103,166,229 

Xn 

-1,0,1 

NAVY 

USN  carrier  battle  groups 
deployed 

mm 

1,2,3 

mm 

-1,0,1 

RSAM 

Red  replacementdaysfor 
„  SAM  available 

§13 

0,2,4 

Xn 

-1,0,1 

Various  precision  guided  munitions  were  represented  collectively  by  the  Joint 
Standoff  Weapon  (JSOW)  variable.  The  munitions  quantity  (shown  by  **  in  Table  2.5) 
and  their  nominal  values  are  as  follows: 


5300 

JDAM1 

1800 

JDAM2 

800 

JSOWC 

800 

JSOWS 

8800 

WCMDC 

1200 

WCMDS 

2100 

HARM 

The  medium  level  for  NAVY  is  two  carrier  battle  groups  (CVBG)  operating  in  each 
theater.  The  nominal  CVBG  will  carry  12  F-14  and  36  F-18C/E.  The  nominal  USAF 
deployment  is  listed  below.  It  includes  tankers,  an  Airborne  Warning  and  Control  System 
(AWACS),  and  a  Joint  Surveillance  Target  Attack  Radar  System  (JSTARS)  (not 
explicitly  played): 
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# 


18 

F-15C  (air-air) 

24 

F-15E  (multi) 

36 

F-16  block  50  (HARM  shooters  for  SEAD) 

12 

A-10 

36 

F-16  block  40  (LANTIRN  capable) 

8 

B-2 

10 

B-l 

10 

B-52 

12 

F-117 

2.3.2  Minimizing  Run  Time 

We  are  interested  in  a  second-order  polynomial  of  the  form 

y~  Po  +  Pl*l  +  ...  +  Pl3*13  +  Pl+t2 1  +  ...  +  p2 6^  13  +  P27*l*2  +  ...  +  Pl04*12Xl3, 

where  each  variable  is  considered  at  three  levels.  At  least  105  design  points  are  required 
to  estimate  the  coefficients  using  least  squares.  For  13  variables  at  three  levels  each,  a  full 
factorial  experiment  would  consist  of  3ft  =  1,594,323  simulation  runs  by  the  JICM.  This 
is  definitely  a  case  for  fractional  factorial  design.  Fractional  factorial  design  procedures 
developed  by  JHU/APL  allow  a  significant  reduction  in  design  runs  to  capture  these 
effects.10  A  design  can  be  created  that  estimates  all  of  the  first-  and  second-order  effects 
with  729  runs.  Of  the  approximately  1.5  million  possibilities,  only  879  cases  were  run 
-  (729  for  experimental  design  and  150  for  validation). 

2.3.3  Measures  of  Outcome 

Twenty  four  (24)  measures  of  outcome  per  theater  were  collected  and  are  listed  in 
Table  2.7. 


Table  2.7.  Complete  listing  of  MOO. 
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Halt  %  area 

USN  AC  start 

Halt  time  (to  1/2  day) 

USN  AC  loss  to  AA 

Red  AC  start 

USN  AC  loss  to  SAM 

Red  AC  loss  to  AA 

USN  AC  loss  to  all  other 

Red  AC  loss  to  SAM 

US  Ground  ED  start 

Red  AC  loss  to  all  other 

US  Ground  ED  loss 

Red  Ground  ED  start 

USAF  AC  start 

Red  Ground  ED  loss 

USAF  AC  loss  to  AA 

Ally  AC  start 

USAF  AC  loss  to  SAM 

Ally  AC  loss 

USAF  AC  loss  to  all  other 

Ally  Ground  ED  start 

Effective  Divisions 

Ally  Ground  ED  loss 

Effective  Division  Attrited 

2.3.4  Stopping  Conditions 

The  halt  phase  ends  when  one  of  three  criteria  is  met:  (1)  Red  reaches  objective 
positions  (Red  win);  (2)  Red  is  attntted  down  to  40%  (Red  loss);  or  (3)  time  expires  at  15 
days  (Red  loss  by  not  achieving  its  objective).  Each  iteration  of  the  simulation  continued 
until  one  of  the  stopping  conditions  was  met. 

There  were  several  choices  for  stopping  conditions  not  used.  Letting  all  battles 
run  through  either  the  defeat  or  success  of  Red  forces  is  a  reasonable  condition  but,  under 
many  of  the  729  starting  conditions,  Red  and  Blue  reach  a  stalemate  that  continues  for 
months;  such  battles  are  a  poor  comparison  to  those  that  last  only  hours. 

2.4  SIMULATION  RESULTS 
2.4.1  Collected  Data  Viability 

The  theaters  considered  are  very  compact.  In  both,  Red  forces  advance  along  a 
single  corridor  and  pursue  a  limited  objective.  No  consideration  was  given  to  the  total 
force  structure  of  the  Department  of  Defense;  only  assets  in  the  theater  were  used,  with 
no  possibility  of  replenishment  or  reinforcement  no  matter  how  long  the  battle  lasted. 
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The  JICM  does  not  model  naval  battles  at  sea  and  represents  land  forces  as 
brigade-sized  units.  This  limited  the  scope  of  modeling  done  for  this  study  by  reducing 
the  fidelity  available. 

2.4.2  Verification 

A  verification  process  was  established  for  the  JICM  output  data.  Output  was 
examined  graphically  and  through  subject  matter  expert  (SME)  review  for  divergence 
from  accepted  norms.  This  procedure  for  verification  is  acceptable  because  the  output  is 
expected  to  conform  to  standard  military  accepted  practices — the  expected  value  outcome 
is  desired.  MOO  dependency  on  input  parameters  was  examined  as  well  as  the  general 
behavior  of  significant  factors  for  each  approximation. 
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3  DEVELOPING  RESPONSE  SURFACES 


3.1  APPROXIMATING  THE  SURFACE 

3.1.1  Measures  of  Outcome  Explained 

The  significant  factors  for  the  primary  MOO  (Halt  Max  %)  are  shown  in  Figure 
3.1.  For  NEA,  a  total  of  nine  factors  are  in  the  response  surface  model,  with  eight 
significant  at  the  0.01%  level  (shown  in  purple).  There  are  10  significant  interactions 
among  the  factors  listed.  For  SWA,  a  total  of  nine  factors  are  in  the  response  surface 
model,  with  six  significant  at  the  0.01%  level  (shown  in  purple).  There  are  14  significant 
interactions  among  the  factors  listed  in  Figure  3.1. 3  (See  Appendix  B  for  the  significant 
interactions  associated  with  the  response  approximations.)  Figure  3.2  shows 
representative  plots  of  interactions  with  strong  influence  and  indicate  factor  behavior 
between  the  two  most  significant  factors  and  the  identified  MOO.  These  plots  are  located 
in  Appendix  B  for  all  of  the  other  MOO. 


NEA  SWA 


Figure  3.1.  Significant  factor  displays. 
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Figure  3.2.  Plots  for  strong  interaction:  (upper)  NEA;  (lower)  SWA. 


3.1.2  Interpreting  n-Space 

The  polynomials  developed  describe  a  polyhedron  and  specific  vertices  along  its 
edge.  This  process  hopes  to  move  along  the  surface  of  each  polyhedron  and  find 
maximum  or  minimum  points.  Because  the  surface  is  in  multidimensional  space,  it  is 
hard  to  visualize  this  search. 

The  experimental  design  matrix  allowed  a  hyper-surface  to  be  constructed.  This 
surface  looks  more  like  a  “net”  than  a  solid  sheet,  however.  Each  of  the  729  runs 
provided  a  new  point  of  intersection  or  node  for  the  strings  of  the  net.  The  more  runs,  the 
more  points  of  intersection,  and  the  more  solid  the  surface  becomes.  A  significant 
drawback  of  fins  process  is  that  the  space  between  these  nodes  is  unmodeled  and  the  true 
behavior  of  the  model  in  that  region  is  unknown.  This  is  not  generally  a  problem  when 
discrete  input  parameters  are  considered,  but  in  this  case  many  discrete,  integer  levels  of 
each  parameter  exist  between  the  minimum,  middle,  and  maximum  values.  There  is  the 
possibility  that  file  model  is  chaotic  or  non-monotonic  in  these  intervals  and  that  this 
behavior  is  undetected  (i.e.,  the  model  will  produce  quadratic  responses  while  the  actual 
system  varies  wildly). 
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3.2  DATA  ANALYSIS 


3.2.1  Low-Dimension  Surfaces 

STATISTIC  A  and  MATLAB  were  used  to  show  simple  graphic  relationships 
between  the  two  most  significant  factors  for  each  MOO.  This  was  confidence  building 
and  allowed  SMEs  to  make  graphic  comparisons  to  accepted  or  intuitive  standards. 
Figure  3.3  shows  one  such  plot  relating  Patriot  missile  batteries  and  Tomahawk  missiles 
(TLAMs)  to  the  number  of  Red  aircraft  lost  because  of  surface-to-air  missiles  (SAMs).  It 
is  expected  that  as  the  number  of  TLAM  increases  there  is  little  effect  on  air  losses  for 
the  enemy.  However,  the  Patriot  batteries  directly  engage  enemy  aircraft  in  flight,  and  as 
their  quantity  increases  there  is  a  steep  slope  in  the  aircraft  lost  MOO.  These  types  of 
comparisons  were  done  for  all  MOOs  as  part  of  the  verification  process. 


■I  above 

Figure  3.3.  Patriot  and  TLAM,  2-D  surface  for  “RED  Aircraft  Lost  to  SAM.”3 
3.2.2  Regression 

It  is  always  necessary  to  examine  the  fitted  model  to  ensure  that  it  gives  an 
adequate  approximation  to  die  true  system  and  that  none  of  the  least  squares  assumptions 
are  violated.  If  this  is  not  done  before  proceeding  with  the  exploration  and  optimization 
of  the  fitted  response  surface,  the  model  will  likely  give  misleading  results.  Several 
regression  methods  are  used  to  accomplish  this,  including  variable  selection  techniques 
and  prediction  criteria.  Often  it  is  difficult  to  resist  the  temptation  to  make  the  model 
overly  complicated;  model  selection  does  not  necessarily  mean  perfect  fit.  The  coefficient 
of  determination,  R2,  measures  a  model’s  capability  to  fit  the  data  (i.e.,  how  far  away 
from  the  predicted  value  is  an  actual  measurement  expected  to  be?).  The  use  of  R2  is 
risky  because  almost  any  addition  of  a  new  model  term  results  in  an  increase  in  R2.  A 
system  modeled  by  every  possible  parameter  will  provide  perfect  results  but  be  very 
complex  and  not  responsive  to  changes.  A  better  criterion  to  use  is  the  adjusted  R2 
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because  it  guards  against  overfitting.  It  punishes  the  user  who  includes  marginally 
important  model  terms  to  gain  accuracy. 

One  would  also  like  to  select  a  model  that  will  best  predict  the  response.  Ordinary 
residuals  are  not  generally  indicative  of  how  the  regression  model  will  predict  as  they  are 
a  measure  of  quality-of-fit  and  have  no  bearing  on  future  prediction.  A  very  important 
criterion,  used  as  a  form  of  model  validation,  is  the  PRESS  statistic.  The  PRESS  residual 
is  a  measure  of  prediction  error: 


PRESSresid  ~  yi  ~  yhati-i 

The  value  yhatiri  is  the  predicted  response  without  observation  i  in  the  model. 
The  data  points  where  prediction  is  poor  is  a  point  where  the  PRESS  residual  is  much 
larger  in  magnitude  than  the  ordinary  residual: 

PRESSsta,  =  Z"™=i(ej>z)2- 

Here  yt  is  not  simultaneously  used  for  fit  and  model  assessment.  For  the  choice  of 
the  best  model,  the  user  would  want  the  smallest  PRESS . 

Another  criterion,  Cp,  deals  with  model  fit.  A  model  that  is  too  simple  may  have 
biased  coefficients  and  prediction.  An  overly  complicated  model  will  result  in  large 
variances  in  the  coefficient  and  in  the  prediction.  We  want  to  choose  a  proper  subset  of 
regressors  so  that  a  suitable  balance  between  overfitting  and  underfitting  is  reached.  The 
Cp  statistic  can  be  an  extremely  useful  criterion  for  discriminating  between  models.  One 
favors  the  candidate  model  with  the  smallest  Cp  value. 

The  Cp  for  a  p  regression  model  is 

Cp  =p  +  (s2  -  c?)(n  -p)/<32. 

Figure  3.4  gives  examples  of  the  statistics  and  how  they  relate  to  the  MOO  %  Halt 
Max  as  discussed.  It  shows  the  MOO  consisting  of  a  number  of  parameters  ip),  MSE 
(mean  square  error),  PRESS,  Adj  R2 ,  and  Cp  for  models  at  each  significance  level.  If  a 
model  contained  a  significant  interaction,  its  main  effects  are  included  in  the  model, 
whether  they  were  significant  or  not.  The  choice  of  polynomial  approximations  came  by 
considering. the  criteria,  significance  level  of  the  parameters,  and  the  size  of  their 
coefficients. 
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Figure  3.4.  Statistics  for  MOO  Halt  Max  %. 

3.2.3  Residuals 

Once  the  approximations  are  derived  for  each  measure  of  output,  diagnostics  help 
detect  unusual  points  in  the  data  set.  Parameter  estimates  and  predictions  may  depend 
more  on  an  influential  subset  of  the  data  than  on  the  majority.  We  want  to  locate 
influential  points  and  assess  their  impact  on  the  model.  Analysis  of  the  residuals  can  be 
used  to  detect  such  things  as  model  misspecification,  departure  from  the  homogeneous 
variance  assumption,  existence  of  suspect  data  points,  and  isolated  high-influence  data 
points.  If  there  are  “bad”  values,  they  should  be  eliminated,  but  if  there  is  nothing  wrong 
with  the  points  except  that  they  control  major  model  properties,  we  would  like  to  know. 

After  running  several  procedures  in  SAS®,  version  8.2,  several  statistics  were 
examined  to  aid  in  detecting  unusual  data  points.  An  outlier  is  a  data  point  that  is  extreme 
in  the  vertical  direction  (see  Figure  3.5).  Detection  of  an  outlier  means  that  an 
observation  does  not  agree  with  the  specified  model.  A  usual  way  to  detect  outliers  is  to 
see  if  any  of  the  residuals  are  very  large  compared  to  the  rest  of  the  residuals.  A  common 
statistic,  which  accentuates  outliers,  used  as  a  diagnostic  in  the  detection  of  outliers  is  the 
i?-Student  statistic  (tf).  An  observation  is  considered  an  outlier  if  its  i?-Student  value  was 
greater  than  3.92  roughly.  Table  3.1  lists  extreme  observations  that  are  considered 
outliers;  if  an  MOO  does  not  have  any  outliers,  it  is  not  listed.  High-influence  points  are 
characterized  by  data  points  that  are  extreme  in  both  the  x  and  the  y  direction. 


3-5 


Figure  3.5.  Residual  plot  with  outliers.5 

A  general  rule  of  thumb  is  that  if  the  magnitude  of  ti  is  significantly  greater  than 
then  we  have  an  outlier  or  high-influence-point.  From  Table  3.1,  the  observations  are 
highlighted  where  the  R-Student  statistics  are  significantly  different  from  the  studentized 
residual.  Since  there  is  not  a  large  discrepancy  between  r,-  and  ff,  we  believe  that  none  of 
the  observations,  even  the  points  considered  outliers,  are  influential.  The  HAT  diagonal 
(hit)  provides  a  measure  of  standardized  difference  from  the  observation  to  the  data  center 
in  the  regressors.  The  R-Student  values  and  HAT  diagonal  values  are  proper  diagnostics 
to  isolate  data  points  that  are  exerting  disproportionate  influence.  A  general  guideline  to 
detect  high-leverage  points  is  if  ha  >  2p/n.  None  of  the  observations  for  any  of  the  MOO 
were  considered  high-leverage  points  because  this  was  not  an  observational  study  but  an 
experimental  design. 
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Table  3.1.  Points  exhibiting  high  leverage.5 


Variable 

NEA 

SWA 

Name 

Observations 

B 

D 

hu 

Observations 

B 

B 

hu 

Red  AC  loss  other 

None 

_ 

m 

_ 

m 

-Mi;;; 

USAF  AC  loss 
other 

661 

-4.20 

-4.25 

.035 

;:po 

■ 

; 

6.27 

Hum 

BH 

n 

671 

4.71 

4.79 

.034 

mil 

■4.89  : 

425 

.030 

None 

— 

_ 

_ 

US  N  AC  loss  AA 

507 

-4.52 

-4.58 

.028 

493 

3.95 

3.99 

.042 

426 

3.87 

3.91 

_ 

B 

— 

Ally  AC  loss 

None 

M 

_ 

60 

4.00 

4.04 

.034 

None 

_ 

w 

_ 

299 

4.91 

4.99 

.025 

None 

_ 

— 

_ 

315 

5.77 

5.90 

.015 

US  Gnd  ED  loss 

484 

4.58 

4.64 

.017 

612 

5.40 

5.51 

.011 

305 

6.11 

6.27 

— 

— 

B 

Ally  Gnd  ED  loss 

305 

3.89 

3.93 

.018 

None 

- 

B 

— 

When  the  studentized  residual  vs.  predicted  y  plots  were  examined,  there  was  no 
real  trend  or  funnel  effect  in  the  scatter.  From  this,  one  can  assume  homogeneous 
variance.  Studying  the  partial  regression  residual  plots  determines  that  none  of  the 
models  are  under-specified  or  in  need  of  a  transformation.  From  the  plots,  influential 
outliers  are  observed  for  USAF  Air  Craft  loss  due  to  other  (see  Figure  3.6).  From  Table 
3.1,  the  most  influential  observations  are  for  USAF  Air  Craft  loss  due  to  other,  these 
points  appear  most  clearly  in  the  SWA  plot  on  the  right  where  the  three  significant 
outliers  listed  in  Table  3.1  are  seen  the  upper  right  portion  of  the  residuals. 
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Figure  3.6.  Residuals  of  USAF  losses  other  than  air  combat  and  SAM. 


3.2.4  Validation 

Another  150  cases  wore  modeled  through  JICM  with  different  factor 
combinations  than  the  729  used  to  develop  the  models.  The  highlighted  values  in  Table 
3.2  indicate  when  the  difference  between  the  two  adjusted  Rz  measures  exceeds  10%, 
when  the  average  absolute  difference  (AAD)  with  respect  to  the  average  of  the  observed 
values  exceeds  10%,  or  when  the  maximum  absolute  difference  (MAD)  with  respect  to 
observed  value  exceeds  20%. 

Twenty-one  of  the  28  models  predict  very  well.  The  seven  exceptions  are 
addressed  in  Table  3.3.  There  seems  to  be  a  problem  with  the  three  response  variables 
dealing  with  the  ground  forces  for  both  theaters.  For  the  others,  a  transformation  of  the 
response  variable  or  the  use  of  weighted  least  squares  may  be  useful,  but  not  necessary, 
because  they  already  have  high  adjusted  R2  values.  Table  3.2  shows  the  response 
variables  with  validation  problems  and  the  associated  reason. 


Table  3.2.  Summary  of  validation  analysis.5 


AAD/Avg 

MAD/Max 

Observed,  % 

Observed,  % 

Halt  Max  % 


Halt  Time 


Red  AC  Loss  AA 


Red  AC  Loss  SAM 


Red  AC  Loss  Other 


AFAC  Loss  AA 


AFAC  Loss  SAM 


AFAC  Loss  Other 


NAC  Loss  AA 


Allied  AC  Loss 


Red  Ground  ED  Loss  6 1 


US  Ground  ED  Loss  74 


Allied  Ground  ED  Loss 


'  -X-~A 


Hah  Max  % 


Halt  Time 


Red  AC  Loss  AA 


Red  AC  Loss  SAM 


Red  AC  Loss  Other 


AFAC  Loss  AA 


AFAC  Loss  SAM 


AFAC  Loss  Other 


NAC  Loss  AA 


NAC  Loss  SAM 


Allied  AC  Loss 


Red  Ground  ED  Loss 


US  Ground  ED  Loss 


Allied  Ground  ED  Loss 


i 
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Table  3.3.  Summary  of  significant  outliers. 


Measure  of  Output 

NEA 

Halt  Time 

Due  to  the  discrete  nature  of  the  outputs,  there  were  large  absolute 
differences  between  the  predicted  and  the  observed 

AFAC  Loss  Other 

Due  to  outliers  (but  still  has  very  high  adjusted  Rl) 

US  Ground  Loss 

Due  to  outliers  and  the  heterogeneous  variance  in  the  model 

Allied  Ground  Loss 

Just  a  very  poorly  fit  model,  something  unexplained  about  the 
system 

SWA 

Red  Ground  Loss 

Due  to  outliers  and  the  heterogeneous  variance  in  the  model 

US  Ground  Loss 

Due  to  outliers  and  the  heterogeneous  variance  in  the  model 

Allied  Ground  Loss 

Just  a  very  poorly  fit  model,  something  unexplained  about  the 
system 

3.2.5  Third-Order  Effects  Explored 

Given  that  there  are  two-way  interactions,  we  examined  three-way  factors  for 
their  importance  in  explaining  the  MOO.  A  two-level  resolution  VII  fractional  factorial 
design  was  developed  so  that  the  three-way  interactions  were  able  to  be  estimated.  The 
1024  runs  were  sent  to  RAND  to  be  run  through  JICM.  With  a  two-level  design,  we  can 
look  only  at  linear  effects,  but  we  already  knew  about  the  curvature  in  the  model  through 
the  previous  three-level  design.  A  complete  three-level  resolution  VII  design  would  have 
taken  20,000  runs.  The  interaction  plots  for  the  most  significant  three-way  interaction  for 
each  measure  of  output  were  produced  and  evaluated. 

Although  it  does  appear  that  there  are  some  significant  three-way  interactions,  the 
second-order  models  already  derived  proved  to  be  equally  accurate  and  were  used  as  the 
response  surface  prediction  equations. 

3.3  SUMMARY  OF  SURFACE  DEVELOPMENT 

Response  surface  methodology  (RSM)  was  used  in  the  attempt  to  accurately 
predict  and  optimize  14  MOO  (responses)  from  the  JICM.  RSM  consisted  of  designing  a 
suitable  experiment  to  evaluate  13  input  variables  that  would  allow  the  user  to  include 
significant  first-  and  second-order  effects  in  the  model.  Several  model  adequacy 
techniques  were  used  to  make  sure  the  model  would  not  be  under-  or  over-specified.  A 
specific  second-order  model  was  chosen  from  those  techniques  for  each  MOO.  The 
polynomial  approximations  for  every  MOO  are  listed  in  Appendix  B.  Exploratory  data 
analysis  was  used  to  check  the  validity  of  the  least  squares  assumptions:  normality  and 
homogeneity  of  variances. 

General  conclusions  and  suggestions  are  as  follows: 
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USAF,  NAVY,  MR,  PAT,  and  HELO  are  the  most  influential  factors  on  the 
response  variables,  which  are  Halt  Max  %,  Halt  Time,  Air  Craft  losses,  and 
Ground  losses. 

Three-way  factors  are  significant  and  should  be  explored  further. 

For  future  studies,  perform  a  two-level  screening  experiment  so  that  more 
factors  can  be  looked  at  instead  of  a  small,  pre-chosen  subset  of  factors.  After 
the  screening  experiment  has  been  performed  and  the  number  of  input 
variables  has  been  reduced,  design  one  experiment  that  can  estimate  both  the 
three-way  interactions  and  curvature  effects  if  it  is  desired  to  estimate  the 
three-way  interactions. 

The  models  produce  better  predictions  in  SWA  than  in  NEA. 

The  MOO  for  ground  forces  lost  are  predicted  poorly  in  both  theaters,  but  the 
majority  of  the  other  models  predict  very  well.  The  poor  prediction  in  some 
cases  is  a  result  of  the  coarse  quantization  of  the  MOO  variables. 


4  OPTIMIZATION 


4.1  CONCEPT  AND  METHODOLOGY 

4.1.1  Optimization  Goals 

The  polynomial  approximations  discussed  previously  provide  a  representation  of 
the  simulation’s  “response”  to  a  set  of  input  values  without  having  to  develop  and 
execute  subsequent  runs.  This  representation  provides  a  rapid,  low-cost  predictive 
capability.  By  themselves,  these  equations  provide  the  capability  to  answer  questions 
such  as,  “For  a  given  force  mix,  what  is  the  expected  outcome  of  a  campaign?” 

What  the  equations  do  not  provide  is  an  easy  way  to  address  questions  of  the  type 
“What  force  mix  is  BEST  used  to  achieve  a  DESIRED  outcome?”  It  is  easy  to  envision 
an  iterative  process  of  “guessing”  a  force  mix,  finding  the  predicted  outcome,  and  then 
adjusting  the  force  mix  until  an  acceptable  outcome  is  achieved.  Such  an  approach  has 
three  major  faults:  (1)  it  is  time  consuming,  (2)  there  is  no  assurance  an  acceptable 
solution  can  be  found,  and  (3)  even  if  a  solution  is  found,  there  is  no  assurance  that  it  is 
the  best  solution. 

4.1.2  Optimization  Formulation 
Linear  Formulation 

The  goal  of  the  standard  form  of  a  linear  optimization  model  is  to  find  the  values 
of  input  parameters  that  minimize  (or  maximize)  an  objective  function:11 


Min  (or  Max) 

Z  c,x,  = 2 

/= i 

n 

s.t. 

Ha,,x,=b,’ 

>1 

Vi 

n 

o 

Al 

V / 

where 

xj  is  the  /th  decision  variable  (factor) 

CLij  is  a  vector  of  coefficients  for  xj 
bj  is  a  vector  of  constraining  values 
Cj  is  a  vector  objective  function  coefficients 

This  a  standard  approach  for  optimizing  systems  of  equations.  If  the  response 
surface  equations  were  linear,  the  above  optimization  formulation  could  be  applied  and 
the  mathematics  of  linear  optimization  could  be  used.  However,  the  response  surface  has 
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second-order  elements  which  do  not  follow  the  format  for  linear  optimization.  Therefore, 
a  non-linear  formulation  was  used. 

Nonlinear  Formulation 

The  objective  function  for  a  nonlinear  system  relies  on  a  more  general 
formulation  where  each  xt  is  allowed  to  be  a  function,  defined  over  all  of  X.  For  the 
specific  problem  this  report  is  addressing,  these  equations  are  the  polynomial 
approximation  fit  equations.  The  formulation  now  is,  find  the  factor  values  that  minimize 
(or  maximize)  the  objective  function,  where12 

Min  (or  Max)  /(*  ,  xt ,  x  , ...,  x)  =  Z 

s.t.  £,  (*,  >  .  * ,  •••>  x  )  (<£,  =  or  *)  ft 

g2(x  ,x  ,x  ,...,x  )  (<,=  or>)  b2 
M 

gm(xt  ,xx,x}  )  ( <,=  or>)bm 

where 

xj  is  the yth  decision  variable  (factor) 

/  is  a  one  or  more  functions  of  xj 

gm  is  a  set  of  functions  constraining  the  objective  function/ 

bj  is  a  vector  of  constraining  values 

4.1.3  Optimization  Model  Description 

In  support  of  this  proof  of  concept,  specific  questions  the  model  might  address  are 
unknown  and  the  basic  design  philosophy  was  to  build  as  many  options  into  the  model  as 
possible.  The  options  are  activated/deactivated  by  modifying  input  data  values  rather  than 
modifying  source  code.  This  allowed  the  research  team  to  quickly  modify  the 
optimization  process  and  generate  new  results.  Each  component  of  the  optimization 
implementation  is  discussed  in  detail. 

Measures  of  Outcome 

The  MOO  are  the  28  polynomial  approximations  shown  in  Appendix  B.  They  are 
divided  into  14  equations  for  each  theater.  This  allows  the  user  to  select  which  MOOs  are 
important  and  allows  for  different  levels  of  importance  in  the  two  scenarios. 

Objective  Function 

Each  MOO  is  assigned  a  weight  (©),  which  allows  each  MOO  to  have  a  variable 
“value,”  meaning  that  the  overall  contribution  of  that  MOO  is  to  be  adjusted  by  the  user. 
The  objective  function  developed  for  this  study  is  the  sum  of  these  weighted  values: 

Z^i^MOO:),  VMOO. 
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The  objective  function  is  used  to  drive  the  search  of  the  response  surface; 
however,  the  actual  value  of  the  objective  function  has  very  little  real-world  meaning. 
The  user  input  weights  can  range  in  value  from  -1  to  +1 .  A  weight  of  zero  eliminates  the 
MOO  from  consideration,  a  positive  weight  leads  to  maximization,  and  a  negative  weight 
leads  to  minimization.  The  magnitude  of  the  weight  is  an  indication  of  relative 
importance  of  that  MOO  compared  to  others.  In  a  typical  optimization  problem,  the 
MOO  of  interest  would  be  weighted  and  all  others  would  be  assigned  a  weight  of  zero. 
An  assigned  weight  of  zero  indicates  that  the  MOO  is  free  to  take  any  value  that  best 
supports  the  minimization  (or  maximization)  of  the  MOO  of  interest. 

Weighting  combinations  of  MOOs  can  lead  to  unexpected  results.  For  example, 
one  could  attempt  to  minimize  the  losses  of  Blue  forces.  The  model  recommends  that  to 
minimize  Blue  losses,  the  commander  should  minimize  the  number  of  forces  employed. 
While  this  leads  to  minimal  losses,  it  also  leads  to  losing  the  conflict. 

Another  complication  with  multiple  MOOs  bang  active  in  the  objective  function 
is  that  the  values  of  the  MOOs  are  not  scaled  with  respect  to  each  other.  Ilns  allows  some 
MOOs  to  dominate  others.  For  example,  the  loss  of  a  few  aircraft  (which  is  counted  at  the 
individual  level)  can  outweigh  the  measure  of  outcome  “percent  of  the  time  Red  achieves 
its  object”  (which  is  scaled  to  0-1).  Great  care  was  taken  to  ensure  that  this  issue  did  not 
negatively  affect  the  research. 

Constraints 

The  model  incorporates  upper  and  lower  bounds  on  each  of  the  MOOs.  While  this 
constraint  is  not  strictly  required  (it  could  be  enforced  by  bounding  decision  factor 
values),  incorporating  it  constrains  the  decision  space  to  a  region  consistent  with  the 
initial  data  generated  from  JICM.  One  basic  constraint  on  all  the  decision  factors  is  that 
the  model  bounds  each  to  the  extreme  values  used  in  the  JICM  runs.  These  constraints  are 
included  in  the  model  to  restrict  the  optimization  to  the  response  surface  region  generated 
by  tiie  JICM  data.  Since  the  model  reads  MOOs  bounds  from  input  tables,  a  user  of  the 
optimization  can  easily  modify  those  boundaries.  By  further  constraining  the  MOO 
boundaries,  the  user  can  set  up  a  problem  that  would  maximize  a  MOO  (e.g..  Red  aircraft 
losses)  subject  to  an  additional  constraint  of  not  allowing  the  conflict  to  exceed  some 
number  of  days  (e.g.,  Halt  time  <  8  days). 

There  are  13  decision  factors  for  each  of  the  scenarios,  which  are  the  independent 
variables  in  the  MOO  equations  and  the  domain  of  the  response  surface.  Each  of  the 
decision  factors  has  been  scaled  to  [-1,+1]  before  the  response  surface  curves  were 
generated.  This  scaling  is  maintained  within  the  optimization  model  and  its  data  tables. 
Therefore,  the  results  of  the  optimization  must  be  “un-scaled”  to  provide  easy-to-interpret 
results.  Some  of  these  are  true  decision  factors  over  which  the  Blue  commander  has  some 
direct  control  (e.g.,  the  initial  Blue  force  mix);  others  are  factors  that  are  implemented  as 
pseudo-decisions  for  modeling  simplicity. 

The  model  includes  a  constraint  that  the  total  forces  committed  to  all  scenarios  (if 
running  simultaneously)  cannot  exceed  the  total  amount  available.  The  Blue  force  may 
also  be  constrained  by  other  factors  unique  to  the  environments)  (e.g.,  the  number  of 
ground-based  aircraft  cannot  exceed  the  airfield  capacity),  force  configuration  and 
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doctrine  constraints  (e.g.,  helicopter  forces  would  not  be  deployed  with  supporting 
ground  forces),  and  physical  constraints  (e.g.,  the  number  of  munitions  used  cannot 
exceed  the  firing  capacity  of  the  number  of  firing  platforms).  Determining  the  values  of 
such  constraints  is  beyond  the  scope  of  this  project,  but  the  optimization  model  includes 
input  tables  to  allow  them  to  be  incorporated.  The  model  also  includes  splitting  the 
decision  factors  into  subcategories  of  platforms  and  munitions  to  allow  for  incorporation 
of  munitions-platform  constraints,  but  these  have  not  been  added  to  the  model. 

Some  of  the  13  decision  factors  included  in  the  model  are  beyond  the  control  of 
the  Blue  commander  (e.g.,  the  Red  force  mix).  There  may  be  cases  when  the  Blue 
commander  can  indirectly  influence  some  of  them  (e.g.,  a  preemptive  strike  against 
weapons  of  mass  destruction  delivery  platforms  or  opening  another  front). 

4.1.4  Optimization  Model 

The  optimization  model  is  a  nonlinear,  multiple  attribute  deterministic 
optimization.  The  implementation  is  relatively  simple.  The  model  maximizes  the  object 
function  Z,  subject  to  a  set  of  linear  and  nonlinear  constraints. 

Max  Z  =  ^((Of  j  *  MOOi^),  over  all  MOOs  and  scenarios 

s.t.  Lower  Bound  MO(\s  <MOOj,s  ^  Upper  Bound  MOOj)S 
Lower  Bound  DFi,s  <DFi>s  <  Upper  Bound  DFi>s 
E(DFi>s)  <  Global  Upper  Bound  DFj,  summed  over  all  scenarios 

4.2  EXPLORING  OPERATIONAL  SITUATIONS 

Two  operational  situations  (OPSITs)  were  developed  to  demonstrate  the  function 
of  this  methodology. 

4.2.1  OPSm  (SWA  Base  Case) 

This  OPSIT  occurs  in  SWA  and  consists  of  Red  forces  invading  Saudi  Arabia. 
The  Joint  Forces  Commander’s  intent  is  to  halt  the  offensive  as  quickly  as  possible  to 
limit  the  terrain  gained  by  Red.  Based  on  available  intelligence  estimates,  the  RED  side  is 
expected  to  commit  the  majority  of  their  forces  to  the  campaign  but  retain  forces  for 
internal  security.  In  addition,  the  RED  commander’s  intent  is  to  move  as  quickly  as 
possible  to  secure  terrain  before  international  pressures  can  be  brought  to  bear.  The  RED 
commander  also  hopes  that  early  success  will  demoralize  the  Blue  forces  while 
encouraging  other  countries  to  support  the  offensive. 

Constraints  are  based  on  the  intelligence  estimate  that  die  RED  forces’  factors 
(RSAM  and  SCUD)  are  set  to  75%  of  maximum.  Based  on  the  Red  commander’s  intent 
of  rapid  movement,  the  Red  movement  rate  is  set  to  maximum.  The  objective  function 
models  the  Blue  commander’s  objective  of  ending  the  operations  quickly,  the 
“Halt_Time”  weight  is  set  to  -1  for  the  SWA  scenario.  All  other  weights  are  set  to  zero, 
eliminating  them  from  consideration. 
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The  model  returned  a  halt  time  of  4.5  days,  which  is  the  minimum  number  of  days 
required  to  halt  the  Red  advance.  This  is  the  bound  of  the  response  surface,  which  is  what 
one  would  expect  of  a  single  MOO  in  the  objective  function.  Table  4.1  gives  the  scaled 
force  structure  that  corresponds  to  this  halt  time.  Note  that  decision  factors  ARTY,  PAT, 
TLAM,  SCUD,  and  RSAM  are  not  included  in  the  MOO  for  “Halt_Time”  in  SWA  (see 
Appendix  B),  so  they  are  uncontrolled  for  this  problem  and  their  values  make  no 
contribution  to  the  objective  function. 


T able  4.1.  Optimum  decision  factors  values  for  OPSIT  1 . 


HELO 

1.00 

ARM 

-1.00 

ATCM 

1.00 

USAF 

0.97 

ARTY 

0.0 

NAVY 

1.00 

INF 

1.00 

MR 

1.00 

PAT 

-0.49 

TLAM 

0.57 

SCUD 

0.75 

RSAM 

0.75 

JSOW 

-1.00 

Table  4.2  contains  the  values  of  all  the  MOOs  for  this  OPSIT.  While  the  model 
predicts  that  the  advance  was  halted  in  4.5  days,  the  Red  forces  were  able  to  penetrate 
67%  of  die  way  to  their  objective.  The  coalition  forces  lost  close  to  100  aircraft  (75  of 
them  U.S.)  and  a  significant  portion  (85%)  of  a  division  (almost  exclusively  GCC). 
Considering  that  die  United  States  entered  the  conflict  with  less  than  400  aircraft  and 
sustained  a  lose  of  almost  20%  of  them  and  the  GCC  entered  the  conflict  with  8  bridges 
and  lost  almost  50%  of  them,  it  is  reasonable  to  assume  that  the  Blue  commander  would 
not  consider  that  halting  the  invasion  in  4.5  days  was  a  successful  campaign. 


Table  4.2.  MOO  values  for  OPSIT  1,  base  case. 


Halt  Max JPer  Cent 

0.67 

Halt_Time 

4.50 

Red_AC_loss_due_to_AA 

32.17 

Red  AC  loss  due  to  SAM 

32.07 

Red  AC  loss  due  to  other 

126.31 

USAF  AC_  loss  due  to  AA 

7.64 

USAF_AC_loss  due  to  SAM 

18.21 

USAF  AC  loss  due  to  other 

26.07 

US_NAC_loss_due  to  AA 

1.21 

US  NAC  loss  due  to  SAM 

21.35 

Ally  AC  loss 

17.92 
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Red_Ground  ED  loss 

3.45 

US_Ground_ED_  loss 

0.01 

Ally_Groimd_EDloss 

0.85 

4.2.2  OPSIT 1,  Excursion  la  (Set  Red  Penetration) 

Based  on  results  from  the  base  case  of  OPSIT  1,  the  commander  directs  an 
excursion  to  examine  limiting  the  depth  of  the  Red  advance  to  50%  of  that  achieved  in 
the  base  case. 

The  model  setup  remains  the  same  as  in  the  base  case,  except  for  setting  a 
constraint  on  the  depth  of  Red’s  penetration.  This  is  done  by  setting  the  upper  bound  of 
“Halt_Max_Per_Cent”  for  the  SWA  scenario  to  0.33  (50%  of  the  previous  0.67). 

The  model  determined  that  tins  set-up  is  infeasible.  When  the  Red  force  elects  a 
movement  rate  of  1.0,  there  is  no  Blue  option  that  can  prevent  the  Red  forces  from 
achieving  at  least  a  33%  penetration  toward  their  objective. 

4.2.3  OPSIT  1,  Excursion  lb  (Limit  Red  Penetration) 

The  Blue  commanders  might  ask  questions  such  as,  “What  is  the  minimum  Red 
penetration  I  can  expect?”  The  next  set  of  options  explored  was  to  use  the  model  to 
determine  the  minimum  penetration  if  the  conflict  were  allowed  to  continue  beyond  the 
4.5  days.  For  this  excursion,  the  objective  was  to  minimize  the  penetration,  subject  to  the 
conflict  not  exceeding  11  days  (the  upper  boundary  for  the  response  curves).  The  weight 
for  the  “Halt_Max_Per_Cent”  for  the  SWA  is  set  to  —1  to  minimize  that  value,  the  weight 
of  “Halt_time”  is  set  to  zero  to  remove  it  from  the  objective  function,  the  upper  boundary 
of  Halt_Max_Per_Cent  is  returned  to  1,  the  upper  boundary  for  the  Halt_Time  for  the 
SWA  is  set  to  1 1  days,  and  the  rest  of  the  model  setup  is  left  unchanged. 

The  Halt_Max_Per_Cent  reported  for  this  setup  was  0.59,  indicating  that  the  Red 
force  achieved  a  59%  penetration  toward  their  objective,  which  occurred  at  4.87  days  into 
the  conflict.  The  decision  variable  values  that  produced  these  results  are  given  in  Table 
4.3.  As  previously  observed,  TLAM  and  RSAM  are  not  factors  in  the  equation  to 
calculate  “Halt_Max_Per_Cent,”  so  they  are  not  under  control  of  the  model  (for  this 
specific  run)  and  their  values  have  no  meaning. 

Table  4.3.  Optimum  decision  factors  values  for  OPSIT  1,  Excursion  lb. 


HELO 

1.00 

ARM 

1.00 

ATOM 

1.00 

USAF 

1.00 

ARTY 

1.00 

NAVY 

1.00 

INF 

1.00 

MR 

1.00 

PAT 

1.00 
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TLAM 

-1.00 

SCUD 

0.75 

RSAM 

0.75 

JSOW 

1.00 

Table  4.4  contains  the  values  of  the  MOOs  for  this  model  run.  It  indicates  that  the 
Blue  forces  loose  13.6  fewer  aircraft,  while  Red  aircraft  losses  rise  by  27.7  aircraft.  The 
Blue  ground  force  losses  are  unchanged,  while  the  Red  ground  forces  losses  dropped 
slightly  (0.03  ED).  By  allowing  the  conflict  to  last  an  additional  9  hours  (4.5  versus  4.87 
days)  and  changing  the  Blue  tactics,  the  Blue  commander  would  be  able  to 
simultaneously 

•  Reduce  the  depth  of  Red  penetration  (67%  versus  59%) 

•  Reduce  Blue  aircraft  losses  (92.4  versus  87.8),  and 

•  Increase  the  attrition  of  Red  aircraft  (190.6  versus  218.2). 

Table  4.4.  MOO  values  for  OPSIT  1 ,  Excursion  lb. 


Halt  Max  Per  Cent 

0.59 

HaltTime 

4.87 

Red_AC_loss_due  to  AA 

36.74 

Red  AC_loss  due  to  SAM 

52.94 

Red  AC  loss  due  to  other 

128.54 

USAF_AC_loss  due  to  AA 

8.34 

USAF_  AC_loss  due  to  SAM 

18.45 

USAF_  AC  loss  due  to  other 

11.43 

USNAClossduetoAA 

1.37 

US_NAC_loss  due  to  SAM 

21.30 

AllyACJoss 

17.93 

RedGroundED  loss 

3.42 

US_Ground  ED  loss 

0.01 

Ally  Ground  ED  loss 

0.85 

4.2.4  OPSIT  1,  Excursion  2  (Limit  Blue  Loses) 

The  Blue  commander’s  review  of  the  base  case  results  also  reveals  the 
unacceptable  level  of  Blue  losses.  This  was  examined  to  find  alternatives  to  reduce  those 
losses.  The  Red  options  were  reset  to  the  base  case  settings.  All  die  user  weights  were  set 
to  zero  except  for  those  related  to  Blue  forces  losses,  which  were  all  set  to  negative 
values  to  create  an  objective  function  to  minimize  Blue  losses  (Table  4.5).  Optimum 
decision  factor  values  are  shown  in  Table  4.6. 


Table  4.5.  Non-zero  weight  for  OPSIT  1,  Excursion  2a. 
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USAF_AC_loss_due_to_AA 

-1 

USAF_AC_loss_due_to_SAM 

-1 

USAF_AC_loss_due_to_other 

-1 

US_NAC_loss_due_to_AA 

-1 

US_NAC_loss_due_to_SAM 

-1 

Ally_AC_loss 

-1 

US_Ground_ED_loss 

-1 

AUy_Ground_ED_loss 

-1 

Table  4.6.  Optimum  decision  factors  values  for  OPSIT 1,  Excursion  2a. 


HELO 

-0.22 

ARM 

-0.31 

ATCM 

1.00 

USAF 

-1.00 

ARTY 

-0.06 

NAVY 

1.00 

INF 

0.34 

MR 

1.00 

PAT 

1.00 

TLAM 

0.63 

SCUD 

0.75 

RSAM 

0.75 

JSOW 

1.00 

Compared  to  die  base  case,  this  excursion  reduces  the  USAF  and  helicopter  forces 
committed  to  the  conflict  while  slightly  increasing  the  amount  of  Armor  and  leaving  the 
ATCM  and  Navy  components  unchanged  (Table  4-7). 

Table  4.7.  Blue  force  structure  for  OPSIT  1,  base  case  vs.  Excursion  2a. 


Base 

Case 

Excursion 

2a 

HELO 

1.00 

-0.22 

ARM 

-1.00 

-0.31 

ATCM 

1.00 

1.00 

USAF 

0.97 

-1.00 

ARTY 

0.0 

-0.06 

NAVY 

1.00 

1.00 

INF 

1.00 

0.34 

MR 

1.00 

1.00 

PAT 

-0.49 

1.00 

TLAM 

0.57 

0.63 

SCUD 

0.75 

0.75 
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RSAM 

0.75 

0.75 

JSOW 

-1.00 

.1.00 

Compared  to  the  base  case,  under  the  conditions  of  Excursion  2a  this  alternative 

•  Extended  the  conflict  almost  a  day  (4.5  versus  5.39  days), 

•  Increased  the  depth  of  Red’s  penetration  (67%  versus  90%), 

•  Reduced  Blue  aircraft  losses  (92.4  versus  71.6), 

•  Increased  Blue  ground  losses  (0.86  versus  0.88  EDs), 

•  Reduced  Red  aircraft  losses  (190.6  versus  178.2),  and 

•  Reduced  Red  ground  losses  (3.45  versus  2.65  EDs). 

MOO  values  are  given  in  Table  4.8. 

Table  4.8.  MOO  values  for  OPSIT  1,  Excursion  2a. 


Halt_Max_Per_Cent 

0.90 

HaltTime 

5.39 

Red  AC Joss  due  to  AA 

33.77 

Red  AC  Joss  due  to  SAM 

58.85 

Red_ACJoss  due  to  other 

85.54 

USAF_ACJossJlueJo_AA 

5.63 

USAF  ACJoss  due  to  SAM 

9.60 

USAF  ACJoss  due  to  other 

7.87 

US  NAC  loss  due  to  AA 

1.41 

US_NACJoss  due  to  SAM 

25.04 

Ally_AC  loss 

22.03 

Red  Ground  JED  loss 

2.65 

US_Ground_ED  loss 

0.04 

Ally  Ground  ED  loss 

0.84 

4.2.5  OPSIT  1,  Excursion  2b  (Force  Scaling) 

As  mentioned,  in  the  way  the  model  is  applied  thus  far  there  is  a  scaling  problem 
when  multiple  MOOs  are  included  in  an  objective  function.  Excursion  2b  illustrates  that 
issue.  The  model  equates  the  loss  of  a  single  aircraft  with  the  loss  of  a  single  ground 
division.  Since  a  notional  division  consists  of  20,000  soldiers  and  a  notional  aircraft 
consists  of  a  crew  of  2,  equating  the  two  values  leads  to  distorted  results.  In  excursion  2b, 
die  loss  of  0.02  additional  EDs  is  offset  by  saving  21  aircraft.  This  implies  that  the  400 
additional  ground  causalities  (20000  •  0.02)  are  equal  to  the  prevention  of  42  (20.8  •  2) 
air  losses. 

Equating  the  number  of  causalities  (one  could  select  a  different  metric)  produces 
the  scale  factor  that  one  ED  equals  10,000  aircraft.  This  can  be  input  to  the  model  by 
modifying  the  weights  used  in  the  objective  function  (Table  4.9). 
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Table  4.9.  Non-zero  weight  for  OPSIT 1,  Excursion  2b. 


USAF_AC_loss_due_to_AA 

-0.0001 

USAF_AC_loss_due_to_SAM 

-0.0001 

USAF_AC_loss_due_to_other 

-0.0001 

US_NAC_loss_due_to_AA 

-0.0001 

US_NAC_loss_due_to_SAM 

-0.0001 

Ally_AC_loss 

-0.0001 

US_Ground_ED_loss 

-1 

Ally_Ground_ED_loss 

-1 

Adjusting  the  weights  to  correct  for  the  scaling  factor  issues  caused  the  model  to 
return  large  components  of  HELO,  ARM,  UASF,  and  TLAM  to  the  Blue  force  mixture. 
As  before,  ARTY,  INF,  and  JSOW  are  not  included  in  the  MOO  equations  (Table  4.10). 
MOO  values  are  given  in  Table  4.1 1 . 


Table  4.10.  Optimum  decision  factors  values  for  OPSIT  1,  Excursions  2a  and  2b. 


2a 

2b 

HELO 

-0.22 

0.81 

ARM 

-0.31 

1.00 

ATOM 

1.00 

1.00 

USAF 

-1.00 

1.00 

ARTY 

-0.06 

■BEa 

NAVY 

1.00 

1.00 

INF 

0.34 

0.93 

MR 

1.00 

1.00 

PAT 

1.00 

1.00 

TLAM 

0.63 

1.00 

SCUD 

0.75 

0.75 

RSAM 

0.75 

0.75 

JSOW 

1.00 

0.0 

Table  4.11.  MOO  values  for  OPSIT  1 ,  base  case  and  Excursions  2a  and  2b. 


OPSIT  1, 
Base  Case 

Excursion 

2a 

Excursion 

2b 

Halt_Max_Per_Cent 

0.67 

0.90 

0.64 

Halt  Time 

4.50 

5.39 

4.91 

Red  AC  loss  due  to  AA 

32.17 

33.77 

28.92 

Red_AC_loss_due_to_SAM 

32.07 

58.85 

44.93 
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Red  AC_loss  due  to  other 

126.31 

85.54 

123.07 

USAF  AC  loss  due  to  AA 

7.64 

5.63 

6.61 

USAF  AC  loss  due  to  SAM 

18.21 

9.60 

18.54 

USAF_AC_loss_due_to_other 

26.07 

7.87 

11.24 

US_NAC_loss_due_to_AA 

1.21 

1.41 

1.03 

US  NAC  loss  due  to  SAM 

21.35 

25.04 

21.3 

Ally  AC  loss 

17.92 

22.03 

17.5 

Red_Ground_ED_loss 

3.45 

2.65 

3.33 

US  Ground  ED  loss 

0.01 

0.04 

0.009 

Ally_Ground_ED_loss 

0.85 

0.84 

0.83 

Total  Blue  casualties 

17,357 

17,743 

16,932 

4.2.6  OPSIT  1,  Excursion  3  (Simultaneous  Theaters) 

In  addition  to  the  SWA  conflict,  the  political  situation  in  NEA  deteriorates  to  the 
point  that  the  probability  of  a  Red  invasion  in  that  second  theater  is  high.  The  Blue  forces 
commander  must  allocate  sufficient  forces  to  both  theaters  but  is  constrained  by  the  total 
available  force  structure. 

The  NEA  intelligence  estimate  is  that,  as  in  SWA,  the  Red  commander’s  intent  is 
to  move  as  quickly  as  possible  to  secure  his  objectives  before  other  countries  become 
involved.  However,  the  NEA  commander  is  not  concerned  with  either  a  second  front  or 
withholding  a  large  reserve  for  internal  population  control. 

Constraints  are  based  on  the  intelligence  estimate;  Red  forces’  factors  RSAM  and 
SCUD  are  set  to  90%  of  maximum.  Based  on  the  Red  commander’s  intent  of  rapid 
movement,  the  Red  movement  rate  is  set  to  maximum.  Since  each  of  the  Red  forces  is 
operating  independently,  they  are  each  allowed  to  commit  all  of  their  force  to  their 
individual  theater. 

The  Blue  forces  commander  is  allowed  to  move  forces  between  theaters,  but  the 
sum  of  forces  committed  to  both  theaters  cannot  exceed  the  total  Blue  force  structure. 
The  model  is  allowed  to  reallocate  only  1/3  of  each  type  of  force  between  the  theaters; 
thus,  each  theater  must  have  a  minimum  force  mix  equal  to  1/3  of  the  total  force.  To 
achieve  this,  the  maximum  value  for  the  sum  of  the  total  forces  assigned  is  set  to  -1. 

The  Blue  commander’s  objective  (and  objective  function)  is  to  win  both  conflicts, 
which  equates  to  halting  both  invasions  short  of  their  objectives.  The  first  question  to  be 
addressed  is,  “Is  there  a  Blue  force  mixture  that  can  achieve  this  combined  objective?” 
To  model  this,  the  weight  of  Halt_MaxJPer_Cent  (the  variable  for  depth  of  Red 
penetration)  is  set  to  —1  for  both  theaters.  The  negative  weight  is  to  minimize  the 
penetration,  and  the  magnitude  of  the  weights  is  the  same  since  failure  in  either  theater 
would  be  considered  a  mission  failure. 

Table  4.12  contains  the  force  assignments  for  the  two  theaters.  Recall  that  a  0 
value  in  this  table  reflects  sending  2/3  of  the  total  force  to  that  theater.  MOO  values  are 
given  in  Table  4.13. 
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Table  4.12.  Optimum  decision  factors  values  for  OPSIT  1,  Excursions  3a. 


NEA 

SWA 

HELO 

-1.00 

0.0 

ARM 

0.0 

-1.00 

ATOM 

0.0 

-1.00 

USAF 

0.0 

-1.00 

ARTY 

0.0 

-1.00 

NAVY 

0.0 

-1.00 

INF 

0.0 

-1.00 

MR 

1.00 

1.00 

PAT 

-1.00 

0.0 

TLAM 

-0.29 

-0.71 

SCUD 

0.90 

0.75 

RSAM 

0.90 

0.75 

JSOW 

-0.80 

-1.00 

Table  4.13.  MOO  values  for  OPSIT  1 ,  Excursion  3a. 


NEA 

SWA 

Halt  Max  Per  Cent 

0.99 

0.98 

Halt  Time 

8.57 

5.99 

Red_AC  Joss  due  to  AA 

128.39 

33.69 

Red  AC  Joss  due  to  SAM 

46.11 

54.2 

Red  AC  loss  due  to  other 

288.94 

93.88 

USAF  AC  loss  due  to  AA 

4.81 

5.52 

USAFACJoss  due  to  SAM 

5.6 

14.57 

USAF  AC  loss  due  to  other 

51.21 

18.63 

US  NAC  loss  due  to  AA 

5.96 

0.40 

US_NAC  loss  due  to  SAM 

10.40 

13.58 

Ally  AC  loss 

29.22 

34.52 

Red  Ground  ED  loss 

2.29 

2.25 

US_Ground_ED  loss 

0.18 

0.06 

Ally  Ground  ED  loss 

1.55 

0.83 

The  model  indicates  that  the  Red  forces  penetrated  99%  of  the  way  to  their  NEA 
objective  and  98%  of  the  way  to  their  SWA  objective  in  8.57  and  5.99  days,  respectively. 
Within  the  resolution  of  this  model,  one  would  have  to  conclude  that  both  Red  invasions 
were  successful.  While  Red  losses  were  significant — 651  out  of  1105  aircraft  (almost 
60%)  and  just  over  2.5  divisions — Blue  losses  were  devastating.  Of  the  229  UASF 
aircraft,  100  were  destroyed;  of  the  837  allied  aircraft,  64  were  destroyed;  of  the  2 
divisions  and  8  separate  brigades,  2.6  equivalent  divisions  were  lost. 

Since  this  is  the  best  outcome  that  can  be  expected  and  it  used  all  the  available 
U.S.  forces,  the  Blue  commander  will  have  to  do  something  outside  of  the  model  to 
achieve  victory. 
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5  CONCLUSIONS 


5.1  CONCLUSION 

This  report  summarizes  research,  evaluates  product  quality,  and  assesses  the 
viability  of  an  RSM  approach.  This  assessment  takes  into  consideration  the  interests  of 
JHU/APL  as  well  as  emerging  technology.  Overall  it  is  an  effort  to  develop  a  statistical 
tool  or  analytical  methodology  appealing  to  military  sponsors  developing  fixture  combat 
technologies. 

The  results  of  this  effort  support  the  diesis  that  simulation  modeling  and  an 
optimization  process  can  be  used  in  concert  to  examine  military  systems,  capabilities,  and 
warfare  areas  to  provide  analytical  insight  into  the  trade-offs  existing  between 
contributing  systems/capabilities  and  warfare  areas.  We  are  able  to  evaluate  questions  of 
interest  to  military  sponsors  and  provide  fast,  analytic  course  of  action  analysis 
considering  every  aspect  of  a  campaign.  The  sponsor  can  request  excursions,  and  these 
can  be  produced  and  evaluated  on-the-spot. 

5.2  LESSONS  LEARNED 

This  type  of  analysis  is  not  cheap;  it  demands  commitment  of  manpower  and 
money  and  an  intellectual  investment  from  experienced  analysts  to  ensure  small  details 
are  not  overlooked. 

5.2.1  Simulation  Modeling 

The  primary  purpose  of  most  simulation  studies  is  the  approximation  of  system 
parameters  with  the  objective  of  identifying  values  that  optimize  specific  system 
performance  measures.  A  simulation  study  consists  of  several  steps,  such  as  experimental 
design,  data  collection,  verification,  validation,  and  data  analysis.  There  are  two  types  of 
simulations:  Terminating  Simulations  and  Steady-State  Simulations.  In  general,  the 
methods  of  analyzing  each  are  different.13  Stopping  conditions  for  die  JICM  produced 
both  types  for  this  research.  The  stopping  conditions  of  the  simulation  must  be  resolved 
to  ensure  a  quality  comparison.  In  this  research,  a  battle  that  ends  after  1  day  is  compared 
to  a  battle  that  was  terminated  at  15  days  without  a  winner.  Examining  entire  campaigns 
magnifies  this  problem.  Simulation,  as  compared  to  closed-form  models,  is  the 
appropriate  approach  to  data  generation  for  a  study  of  this  type.  It  provides  large  and 
diverse  data  sets  but  requires  the  investment  of  set-up  time  to  establish  the  starting 
parameters. 

5.2.2  Response  Surface  Methodology 

Developing  response  sxirface  approximation  eqxiations  that  allow  the  analyst  to 
exhaustively  map  the  measures  of  outcome  was  beneficial  in  this  study  because  it 
provided  a  high-fidelity  perspective  of  the  expected  response.  However,  RSM  is 
cumbersome  and  has  limited  scope.  The  approximations  are  based  only  on  the  starting 
assumptions.  If  any  of  the  input  parameters  change  in  performance,  structure,  or  quantity 
(outside  of  the  established  bounds),  the  experimental  design  is  in  jeopardy. 
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5.2.3  Trade  Space  Analysis 

This  is  the  right  way  to  look  at  the  problem  but  it  is  driven  by  fidelity.  Our 
knowledge  of  the  trade  space  comes  from  defining  nodes  within  the  space;  the  closer  the 
nodes,  the  better  our  knowledge  of  how  to  move  between  (or  trade)  their  values.  A  sparse 
trade  space  representation  has  a  high  error  in  extrapolation  between  the  nodes.  A  dense 
trade  space  has  low  error  but  intense  computational  requirements. 

5.2.4  Data  Production 

JICM  is  too  aggregate  a  model  for  analysis  of  specific  warfighting  systems. 
Models  used  in  the  future  must  be  able  to  represent  individual  combat  platforms  (a 
collection  of  systems)  but  operate  with  a  strategic  or  global  view. 

An  appropriate  scaling  for  comparison  of  MOO  must  be  developed  and  agreed 
upon.  An  arbitrary  scaling  method  was  presented  in  this  report  but  a  military  sponsor  may 
opt  for  a  much  different  base  comparison  of  asset  value. 

5.2.5  Optimization 

The  algorithms  and  GAMS  code  used  for  this  research  provided  useful  results  that 
are  applicable  within  the  limited  scope  of  the  campaigns  analyzed.  Conceptually  the 
optimization  process  works,  but  the  application  needs  further  research  to  develop  a 
broadly  applicable  methodology.  The  method  employed  here  was  labor  intensive. 

5.3  THE  NEXT  STEP 

5.3.1  Broader  Application 

This  methodology  is  viable  for  examining  warfare  capabilities.  The  logical 
extension  of  this  process  is  its  application  in  a  much  broader  study.  Future  application 
must  consider  diverse  input  parameters  and  more  MOO.  For  example,  larger  operational 
theaters  with  more  complicated  movement  networks,  the  effects  on  strategic  lift 
capabilities,  naval  deep  water  engagements,  and  multiple  simultaneous  theaters  of  war 
can  be  examined. 

5.3.2  Model  Construct 

Response  surface  methodology  is  used  here  to  generate  a  mathematical 
representation  of  each  MOO.  This  process  helped  both  in  the  understanding  and 
explanation  of  platform/system  interaction.  RSM  is  a  cumbersome  task  and,  although  it 
yields  a  vision  of  the  entire  solution  space,  it  is  not  necessary  to  understand  the  entire 
response  mapping  for  all  MOEs.  Future  research  could  take  advantage  of  other 
techniques. 

The  most  productive  future  course  of  action  seems  to  be  integrated  optimization. 
Future  work  using  this  approach  should  combine  simulated  model  results  with  an 
optimization  process  in  a  feedback  loop  so  that  a  single  simulation  run  feeds  the 
optimization  and  the  optimization  dictates  input  values  for  successive  runs.  Here  the 
optimization  process  trades  interim  results  with  the  data-producing  software  to  make 
logical,  improving  changes  to  parameter  values  for  each  iteration.  In  this  way,  various 
starting  points  may  be  examined  in  the  solution  space  with  each  progressing  toward  an 
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optimal  allocation  of  assets  based  on  the  optimization.  Thus,  instead  of  mapping  the 
entire  response  surface  for  each  MOE  and  then  evaluating  the  union  of  all  surfaces  to  find 
a  feasible  region,  a  direct  path  is  constructed  to  an  optimal  solution.  A  starting  point  must 
be  established  using  this  methodology  and  care  must  be  taken  to  avoid  local 
maxima/minima  that  are  suboptimal.  It  is  recommended  that  multiple  starting  points  be 
used  within  the  region  of  interest  and  that  an  aggressive  perturbation  process  be 
developed  to  move  beyond  suboptimal  points. 


5.3.3  Soft  Factors 

Logistics,  communications,  morale,  and  command  are  very  hard  to  model,  and 
many  developers  are  trying  to  incorporate  these  types  of  factors  into  their  models.  These 
can  be  analyzed  using  the  same  methodology  presented  in  this  report  once  adequate 
models  for  these  factors  are  available. 
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PARTIAL  JICM  OUTPUT  DATA  PILES 
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Figure  A.2.  Representative  NBA  numbers  with  summary  statistics. 
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Appendix  B 

POLYNOMIAL  APPROXIMATIONS 

NEA 

Model  (With  coded  variables) 

NBA  Halt  Max  % 

Adj  R2  =  79.89% 

.8197+.0768MR-.0531HELO-.0269ARM-.0162ATCM-.0134USAF-.0123ARTY-.0113NAVY-.0110INF-.0064PAT- 

.0351MR*HELO-.0208MR*ARM-.0103MR*USAF-.0125MR*ATCM-.0082MR*ARTY-.0102MR*INF- 

■0080MR*NAVY+.01 30ARM2 

NEA  Halt  Time 

Adj  RJ  =  60.51% 

1 2.571-1 .293MR+.774ARM+.594USAJ+.553ATCM+.361  PAT- 
.31 1HELO+.041NAVY+.3076INF+1.401MR*HELO-.614ARM*HELO 
.617MR*ARM+.523MR*ATCM+.330MR*NAVY-.279ARTY*HELO+.307MR*INF-.745HELO2-.507MR2- 

•393ARM2 

NEA  Red  AC  loss  due  to  AA 

Adj  R2  =  94.84% 

112.69-12.58TLAM-11.01PAT+9.96NAVY-7.48USAF+1.53RSAM-.76MR+2.41TLAM*USAF- 
1  84PAT*TLAM+1 ,33SCUD*USAF-1 ,22TLAM*RSAM-1 ,20SCUD*PAT- 
1 .06TLAM*NAVY+.99MR*HELO+5.97TLAM2+6.90USAF2 

NEA  Red  AC  loss  due  to  SAM 

Adj  R1  =  9634% 

79.85+25.53PAT+4.02TLAM-2.50USAF-2.30NAVY-2.08MR+.79ARM+.80ATCM+.10HELO- 

.43SCUD+1.96MR*HELO+1.75PAT*TLAM-1.78PAT*USAF-1.33PAT*NAVY+.86SCUD*USAF+3.46USAF2- 

2.89PATI-1 .50TLAM2 

NEA  Red  AC  loss  due  to  other 

Adj  R2  =  81.70% 

322.59+29.53USAF-14.80TLAM-10.93MR+6.40ARM+4.60ATCM-3.27SCUD+2.83INF-3.46NAVY+ 
10.98MR*HELO+5.72MR*  ARM -4.27  ARM*HELO-4.90TLAM*USAF+4.82MR*ATCM+2.95MR*NAVY- 
1 3.89USAF2-10.92TLAM2  -7.16HELO2-5.03MR2 

NEA  USAF  AC  loss  due  to  AA 

Adj  R2  =  96.54% 

4.87+1. 12USAF-.1 03SCUD-.043TX.AM-.039MR+.021RS  AM-.010P AT-.004HELO- 
.180PAT*USAF+.177SCUD*USAF-.132TLAM*USAF-.059PAT*TLAM+.044MR*HELO+.041USAF*RSAM- 

.028USAF*NAVY-.437USAFI+.042TLAM2 

NEA  USAF  AC  loss  due  to  SAM 

Adj  R2  =  98.13% 

5.22+1.66USAF-.775NAVY.607RSAM-.102JSOW+.067PAT-.057SCUD+.268USAF*RSAM-.211NAVY*RSAM- 

.102JSOW*RSAM-.079JSOW*USAF+.050JSOW*NAVY-327RSAM1-.081USAF2+.013NAVYI+.102JSOWI 

NEA  USAF  AC  loss  due  to  other 

Adj  R2  =  93.97% 

24.32-13.64PAT+1 1 .68SCUD+3.90USAF-2.52MR+1 .68  ARM  .84ATCM  -.41HELO+.62INF- 
6.45SCUD*PAT+2.74PAT*USAF+2.29SCUD*USAF+1 ,68MR*PAT-1 ,40ARM*HELO+1  -32MR*ARM- 
95MR*SCUD-1  1 1  PAT* ARM  95SCUD*ARM+1.00MR*ATCM+.75MR*NAVY+.74ARM*USAF+2.93PAT2- 

1 .35HEL02-!  .54USAF2 

NEA  US  NAC  loss  due  to  AA 

Adj  R2  = 99.12% 

5.1(H-2.99NAVY-.71TLAM-.62PAT-.37USAF+.04RSAM-.039MR-.398TLAM*NAW-.326PAT*NAVY- 
.1 99USAF*NAVY-.095PAT*TLAM+.072PAT*USAF-.073SCUD*PAT+.259USAF1-.  156NAVY2 

NEA  US  NAC  loss  due  to  SAM 

Adj  R2  =  99.47% 

9.86+3.62NAVY+1 .39RSAM-.48USAF+.324NAVY*RSAM-.249USAF*RSAM-.l  20USAF*NAVY  -.874RSAM2- 

.794NAVY2 

NEA  Ally  AC  loss 

Adj  R2  “98.49% 

27 .56-1 .99USAF-3.93NAVY+4.58RSAM-.90TLAM-.76PAT-.642MR+.  1 73  ATCM+.06HELO-1 .76N  AVY*RSAM- 
.869USAF*RSAM+.491USAF*NAVY+.321MR*HELO+.499NAVY2+1.07USAF2+.447TLAM-3.09RSAM2 

NEA  Red  Ground  ED  loss 

Adj  R*  “  60.92% 

2.705+.180HELO-.0415MR+.382ATCM+.034NAVY+.015ARM+.019JSOW+.007USAF+.017ARTY+.002RSAM- 
.015PAT+.015SCUD+.042MR*HELO+.034MR*ARM+.023ARTY*RSAM+.033HELO*USAF  .037MR*USAF- 
.017ATCM*USAF-.025HELO*JSOW+.021PAT*HELO-.019SCUD*HELO-.126HELO,-.059MR2 

NEA  US  Ground  ED  loss 

Adj  R2  =  73.64% 

•094+.040MR-.036HELO-.009ATCM-.008NAVY+.008ARTY-.006PAT-.006ARM-.016MR*HELO- 

•008MR*ATCM-.006MR*USAF-.013MR2+005HELO^ 

NEA  Ally  Ground  ED  loss 

Adj  R*  =  64.86% 

1.32-.1 10HELO-.070ARM-.042ATCM-.036USAF-.027NAVY-.019PAT-.019ARTY+.008MR-.015INF- 
064MR*HELO-.052MR*ARM+.028ARM*HELO-.024MR*ATCM-.015MR*NAVY- 
-014MR*ARTY+.032HEL02+.027ARMl 

B-l 


SWA 


Model  (With  coded  variables) 


SWA  Halt  Max 
% 

Adj  R2  =  93.59% 

.8197+.147MR-.  149HELO  -.017ARM-.049ATCM-.028USAF+.003 JSOW-  .008ARTY-.010NAVY-.008INF-.004PAT+  004  SCUD- 
-028MR*HELO-.010MR*ARM+.010MR*USAF+.009MR*ATCM-.006HELO*JSOW-.007ARM*ARTY+.009ARM*HELO- 
.014HELO*ATCM+.009PAT*TLAM-.008SCUD*PAT-.065MR2+.027HEL02-.009JSC)W2-.012ATCM2 

SWA  Halt  Time 

Adj  R2  =  86.64% 

6.35-1 .26HELO-.607MR-.41 8  ATCM-.252USAF  .158ARM-.093NAVY+.062JSOW+.072INF+.620MR*HELO-.193ARM*HELO 
.1 65MR*US  AF+.  1 36MR*  ARM .  1 02MR*  ATCM .  1 05JSOW*USAF-.079MR* JSOW-.09 1HELO*ATCM-.083INF*HELO+.1 78HEL02 

SWA  Red  AC 
loss  due  to  AA 

Adj  R2  =  9438% 

30.26-3.92PAT+3.32NAVY-3.13TLAM-2.55USAF+.536RSAM+.231SCUD+.758PAT*USAF-.620TLAM*RSAM-.610USAF*NAVY- 

.3 1 5TLAM*USAF+2.98TLAM2+1 .59PAT2+.427USAF2 

♦ 

SWA  Red  AC 
loss  due  to  SAM 

Adj  RJ  =  93.97% 

4 1 .22+1 1 .63PAT-5.82US  AF-1 .46TLAM+.585SCUD-.076HELO-2.60PAT*TLAM-.64 1  SCUD*USAF- 
.057SCTJD*HELO+.461MR*PAT-.453MR*ATCM  -.479MR+HELO  4.10USAF2-2.93PATJ+1.31TLAM2 

SWA  Red  AC 
loss  due  to  other 

Adj  R2- 91.23% 

1 1 6.54+1 8. 44USAF -6.86HELO-4.63TLAM-3 . 16MR-2.29ATCM-1 .91 PAT- 
1 .1 6SCUD+1 .608RSAM+3 .37MR*HELO+2.24HELO*RSAM+l  .69SCUD+USAF- 
1 .47P  AT*  USAF+1 .04MR*  ATCM+1 . 1 7HELO*TLAM+l  •14TLAM*RSAM-.881HELO*USAF-8.2USAF2+4  53TLAM2 

+3.35RSAM2+1 .62HEL01 

SWAUSAFAC 
loss  due  to  AA 

Adj  R2  =  89.22% 

8.07+1. 05USAF-.198SCUD-.278TLAM+.091RSAM-.488PAT-.224NAVY-.368PAT*USAF+.222SCUD*USAF-.384TLAM*tJSAF- 

.  123PAT*TLAM-.  1 06TLAM*RSAM+.  1 02US  AF*RSAM-.  1 20USAF*NAVY+.  1 95SCUD*PAT+.  121  PAT*NAVY- 

1 .403.USAF2 +.278TLAM2+.247PAT2 

SWAUSAFAC 
loss  due  to  SAM 

Adj  R2  =  99.09% 

16.06+4.67USAF-2.30NAVY+3.31RSAM-.154SCUD+.148PAT-1.37NAVY*RSAM+.752USAF*RSAM-.767USAF*NAVY- 

2.04RSAMI+.549NAVY2-.580USAF2 

SWAUSAFAC 
loss  due  to  other 

Adj  R2  =  96.75% 

14.96-8.51PAT+8.14SCUD+1.20USAF-1.08MR+.274ARM-.737ATCM-2.17HELO+.734RSAM-4.27SCUD+PAT- 

.529PAT*USAF+.737SCUD*USAF-.503MR*SCUD-.469ARM*HELO+J93MR*PAT+1.31MR*HELO+.969PAT*HELO- 

.966SCUD*HELO- 

.458SCUD*ATCM+.409PAT*ATCM+.346USAF*RSAM+.617HELO*RSAM+.358SCUD*RSAM+.682RSAMltl  34PAT2+ 909HELO2- 

■903USAB3 

SWA  US  N AC 
loss  due  to  AA 

Adj  R2  =  9835% 

.788+.006MR+.566NAVY-.147USAF-.102TLAM-.103PAT+.014HELO+.009RSAM-.092USAF*NAVY-.064TLAM*NAVY- 

.058PAT*NAVY+.037PAT*USAF-.016MR*HELO+.011USAF*RSAM+.093TLAM2+.071USAF2-.040NAVY2+ 

,034PAT*+.01 8HELO2+.016MR2 

SWA  US  N  AC 
loss  due  to  SAM 

Adj  R2  =  99.21% 

16.62+5.53NAVY+3.33RSAM-1.14USAF-.703USAF*RSAM+.598NAVY*RSAM-.470USAF*NAVY-2  120RSAM2- 

526NAVY2+.313USAF* 

SWA  Ally  AC 
loss 

Adj  R2  =  97.59% 

22.75-2.41USAF-4.S0NAVY+4.01RSAM-.209TLAM-.215PAT-.521MR+.512HELO-1.76NAVY*RSAM- 
.709USAF*RSAM+.457USAF*NAVY+.325MR*NAVY-.400MR*RSAM+.192MR*USAF-.197HELO*TJSAF- 
-268HELO*NAVY+l  .1 2NAVYl+l  .23US  AJF2-2.87RSAM2 

SWA  Red 
Gronnd  ED  loss 

Adj  R2  =  67.48% 

2.S1+.277HELO+.120RSAM+.127ATCM+.042USAF- 

.051MR+.039ARM+.154MR*HELO+.149HELO*RSAM+.076ATCM*RSAM+.066MR*ATCM-.054ARM*HELO- 
•048HELO*USAF+.039USAF*RSAM+.l  17RSAM2 

SWAUS 
Ground  ED  loss 

Adj  R2  =  65.78% 

.0296-.0232HELO+.0096MR-.0080USAF-.0074ATCM-.0039PAT-.0029NAVY.0027HELO*USAF+.0032PAT*USAF+.0107HEL02 

SWA  Ally 
Gronnd  ED  loss 

Adj  R2- 52.13% 

.8375-.039HELO+.026MR-.019ATCM-.009USAF-.0022ARM-.0008TLAM+.042MR*HELO+.017MR*ATCM+.010MR*USAF- 

.01 0ARM+TLAM-.025MR2 

Appendix  C 

ACRONYMS  AND  ABBREVIATIONS 


AAD 

average  absolute  difference 

APC 

armored  personnel  carriers 

AWACS 

Airborne  Warning  and  Control  System 

C4ISR 

command,  control,  communication,  computers,  intelligence, 

surveillance,  and  reconnaissance 

CVBG 

aircraft  carrier  battle  group 

ED 

effective  divisions 

GAMS 

General  Algebraic  Modeling  System 

GCAM 

General  Campaign  Analysis  Model 

GCC 

Gulf  Cooperation  Council 

TFV 

Infantry  Fighting  Vehicles 

JHU/APL 

The  Johns  Hopkins  University  Applied  Physics  Laboratory 

jicm 

Joint  Integrated  Contingency  Model 

JSOW 

Joint  Standoff  Weapon 

JSTARS 

Joint  Surveillance  Target  Attack  Radar  System 

JWARS 

Joint  Warfare  System 

MAD 

maximum  absolute  difference 

MOE 

measures  of  effectiveness 

MOO 

measures  of  outcome 

mph 

miles  per  hour 

MSE 

mean  square  error 

? 


NEA 

Northeast  Asia 

NLP 

nonlinear  programming 

opsh 

operational  situation 

RSM 

response  surface  methodology 

SAM 

surface-to-air  missile 

SME 

subject  matter  expert 

SWA 

Southwest  Asia 

TEAM 

Tomahawk  land-attack  missile 

USAF 

U.S.  Air  Force 

USN 

U.S.  Navy 

c-s 


